1. Field of the Invention
In general, the present invention relates to interactive electronic detection and display systems where the motions of a viewer are translated into interactive controls that enable the viewer to manipulate a forward and/or vertically projected three-dimensional image. More particularly, the present invention relates to detection and display systems that translate user/viewer movements into control commands for projected three-dimensional images in a virtual environment.
2. Prior Art Description
Many people experience virtual environments through the playing of a video game. Video games present a virtual environment in which the action of the video game takes place. Typically, a person playing a video game is presented with a control module so that they can control various elements within the virtual environment. If the video game is played with a gaming console, the control module is a handset that communicates with the gaming console. If the virtual environment is presented on an electronic device, such as a smart phone or a tablet computer, then the control module is typically the touch screen or other controls that are built into the electronic device.
As the programming of virtual environments has evolved, motion sensor controllers have been developed so that people can interact with a virtual environment in a more natural manner. Motion sensor controllers detect movements of a person or an object using cameras, reflected infrared light, reflected ultrasonic sound waves, and/or other methods. Detected movements are converted into command signals. In this manner, a handheld control module need not be used. An example of a prior art motion controller is the Kinect® controller system that is used by Microsoft® with its Xbox® game consoles.
Most virtual environments are viewed two-dimensionally. That is, all the graphics contained within the virtual environment are designed to be viewed on a traditional two-dimensional LED display or LCD display with no three-dimensional effects. Consequently, most controllers, be they handheld controllers or motion controllers, are also designed for a virtual environment that is viewed two-dimensionally.
A challenge arises when a controller is needed to interact with a virtual environment that is designed to be viewed three-dimensionally, rather than two-dimensionally. Virtual environments that are viewed three-dimensionally can be generated using many methods. Most methods involve imaging an environment from two stereoscopic viewpoints. The images are superimposed and are varied in color, polarity and/or otherwise to vary the views observed by the left and right eyes of a viewer. For the images to appear to be three-dimensional, the images have to be viewed with specialized 3D glasses or viewed on an auto-stereoscopic display. In this manner, different aspects of the image can be perceived by the left and the right eyes, therein creating a three dimensional effect.
The controls used to manipulate a 3D object in a 3D virtual environment require that a controller have the capability to create movement in the x-axis, y-axis and z-axis. In U.S. Patent Application Publication No. 2012/0117514 to Kim, a prior art system is disclosed where a person's hand is used as a 3D controller. The hand is imaged with a camera and the image is projected into a virtual environment. The system is limited to virtual environments where the 3D virtual environment appears to exist below or behind the display screen.
In U.S. Patent Application Publication No. 2014/0015831 to Kim, the control accuracy of the earlier system is improved by adding a physical object that the hand can touch. Movements to the physical control object are perceived as control signals to control a virtual object in a virtual environment. In order for the physical control system to work well, the physical control object being manipulated has to be generally the same shape and size as the virtual object. Thus, the system is not very dynamic because only a limited number of physical control objects exist.
Using certain techniques that are disclosed in co-pending patent application Ser. No. 15/481,447 to Freeman et al., virtual 3D images can be created that appear to be above or in front of the surface of the screen from which they are displayed. When a virtual 3D image appears in front of or above a display, a viewer may be tempted to reach out and touch the 3D image. The 3D image being displayed is virtual, so this cannot be done. However, the applicant herein presents a system where a 3D image that is projected forward or above its display can be finely manipulated and controlled by interacting with the space where the virtual 3D image appears to be. The details of this improved control system are described and claimed below.